The present invention relates to computer software. The invention relates more specifically to approaches for improving the performance of interactive speech applications.
Computer-based interactive speech applications provide automated interactive communications. For example, a computer-based interactive speech application may be used in a telephone system to automatically answer an incoming call and engage in a dialogue with the caller in order to route the call, provide requested information, or process a transaction. Using speech recognition technology, the application is able to convert a caller""s speech into a textual representation and thereby understand what the caller is saying. These applications are also sometimes categorized under the general heading of interactive voice response (IVR) applications. Where they involve the use of speech recognition technology, these applications are defined here under the more narrow term, xe2x80x9cinteractive speech applicationsxe2x80x9d.
In the past, developing interactive speech response applications that use speech recognition technology has been difficult for the enterprises that implement these applications, their programmers, and others. The software development tools and application testing tools available for use in the development process have been less than satisfactory. One recent significant improvement in the development process involves the use of re-usable software components, commercially known as DialogModules(trademark), that a developer may interconnect to produce effective speech applications quickly. This modular approach is described in co-pending U.S. patent application Ser. No. 09/081,719, filed May 6, 1998, entitled xe2x80x9cSystem and Method for Developing Interactive Speech Applications.xe2x80x9d
Although the modular approach represents a significant advance in the development process, there is still a need for an effective way to determine whether a completed interactive speech application is working effectively. Generally, a developer or programmer prepares an interactive speech application by hand-writing source code, assembling pre-defined objects, or joining modular components using a system such as DialogModules(trademark). The developer compiles the program, installs it on a test system, and verifies that it operates correctly in response to test telephone calls. The program may be debugged and rewritten over a period of time. The completed application is then launched and used on a xe2x80x9clivexe2x80x9d basis.
Even though a completed application operates correctly, meaning that there are no errors in the program code, it may not perform in an optimal manner in a live environment. Detecting performance problems is difficult. Performance problems include a variety of issues or defects, such as the inability of repeated callers to understand a particular prompt or option, callers becoming xe2x80x9clostxe2x80x9d in the logical flow of the application, etc. In the past, developers have received feedback on such problems in a manual way, such as by callers calling an institution to complain about its voice response system.
Thus, there is a need for an automated way to provide feedback on the usability or effectiveness of a completed application.
In addition, there is a need for tools that can be used to identify potential problems with particular components of an application, such as vocabulary, prompts, and call flow.
There is also a need for a mechanism for tuning an application based on its past performance.
There is a further need for a mechanism for improving performance of an application, in which audible prompts and actual caller utterances may be reviewed in conjunction with events that caused such utterances.
The foregoing needs, and other needs and objects that will become apparent from the following description, are fulfilled by the present invention, which comprises, in one aspect, a method of improving performance of an interactive speech application. Generally, the method involves the computer-implemented steps of storing in a log, event information that describes each task carried out by the interactive speech application in response to interaction with one or more callers; displaying an analytical report about selective actions taken by the interactive speech application while executing and about selective actions taken by one or more callers while interacting with the interactive speech application; and modifying the interactive speech application based on the analytical report.
In one feature of this aspect, displaying an analytical report further comprises retrieving a portion of the event information that is associated with a particular call within the log; and for each event represented in the portion of the event information, identifying an event and reporting analytical information concerning the event.
According to another feature, retrieving a portion of the event information comprises receiving a call number that identifies the particular call and skipping over the event information until the portion is located in the log. Still another feature involves reading a record from the log; identifying a call code within the record; and retrieving the record only if the call code corresponds to the particular call. The feature of retrieving a portion of the event information further comprises retrieving one or more records that each contain an event identifier referring to one or more events.
In another feature, displaying an analytical report further comprises identifying an event and reporting analytical information concerning the event, in which the analytical information is selected so as to report one or more potential performance problems in the interactive speech application.
According to another feature, storing a log further comprises, for each of the selective actions mentioned above, storing a record that comprises one or more pairs of keywords and associated values. The keywords each comprise an event code associated with a particular one of the selective actions. The associated values each comprise a description of one of the selective actions.
According to still another feature, the method further comprises storing one or more sets of audio information, in which each of the sets of audio information is associated with one or more utterances by one of the callers. Each of the sets of audio information is also associated with one of the tasks represented in the log.
In another feature, while displaying the analytical report, and upon retrieving event information from the log for one of the tasks that is associated with one of the sets of audio information, the method involves audibly replaying such audio information, pausing the displaying of the analytical report and determining whether to replay such audio information an additional time.
The invention also encompasses a computer data signal in the form of a carrier wave that carries instructions configured according to the foregoing steps and features. The invention also involves an apparatus for carrying out the foregoing steps, as described below.